Legislation is an inherently human endeavor. But just as organizations across industries are unlocking new capabilities and efficiencies through artificial intelligence (AI), governments also can aid their legislative processes through the application of AI. For the past five years, we've studied the potential impact of AI on government. We've looked at everything from how much time AI could save workers in each US federal agency to the rate of AI adoption in US federal, state, and local governments.2 While AI can help many different areas of the legislative process--from AI assistants answering members' questions about legislation to natural language processing analyzing the US Code for contradictions--two key applications stand out.

This mantra implies that human stakeholders, like a domain expert or data analyst, could leverage visual analytics techniques to seek answers to known unknowns and discover unknown unknowns in the course of the data sense-making process. We argue that in the era of AI-driven automation, we need to recalibrate the roles of humans and machines (e.g., a machine learning model) as teammates. We posit that by realizing human-machine teams as a stakeholder unit, we can better achieve the best of both worlds: automation transparency and human reasoning efficacy. However, this also increases the burden on analysts and domain experts towards performing more cognitively demanding tasks than what they are used to. In this paper, we reflect on the complementary roles in a human-machine team through the lens of cognitive psychology and map them to existing and emerging research in the visual analytics community.

Humans have been teaming up with machines throughout history to achieve goals, be it by using simple machines to move materials or complex machines to travel in space. But advances in artificial intelligence today bring possibilities for even more sophisticated teamwork--true human-machine teams that cooperate to solve complex problems. Much of the development of these human-machine teams focuses on the machine, tackling the technology challenges of training AI algorithms to perform their role in a mission effectively. But less focus, MIT Lincoln Laboratory researchers say, has been given to the human side of the team. What if the machine works perfectly, but the human is struggling?

Humans have been teaming up with machines throughout history to achieve goals, be it by using simple machines to move materials or complex machines to travel in space. But advances in artificial intelligence today bring possibilities for even more sophisticated teamwork -- true human-machine teams that cooperate to solve complex problems. Much of the development of these human-machine teams focuses on the machine, tackling the technology challenges of training AI algorithms to perform their role in a mission effectively. But less focus, MIT Lincoln Laboratory researchers say, has been given to the human side of the team. What if the machine works perfectly, but the human is struggling?

Because in military situations, as well as for self-driving cars, information must be processed faster than humans can achieve, determination of context computationally, also known as situational assessment, is increasingly important. In this article, we introduce the topic of context, and we discuss what is known about the heretofore intractable research problem on the effects of interdependence, present in the best of human teams; we close by proposing that interdependence must be mastered mathematically to operate human-machine teams efficiently, to advance theory, and to make the machine actions directed by AI explainable to team members and society. The special topic articles in this issue and a subsequent issue of AI Magazine review ongoing mature research and operational programs that address context for human-machine teams. In 1983, William Lawless blew the whistle on Department of Energy (DOE) mismanagement of military radioactive wastes. After his PhD, he joined DOE's citizen advisory board at its Savannah River Site where he coauthored over 100 recommendations on its cleanup.

A rising vision for AI in the open world centers on the development of systems that can complement humans for perceptual, diagnostic, and reasoning tasks. To date, systems aimed at complementing the skills of people have employed models trained to be as accurate as possible in isolation. We demonstrate how an end-to-end learning strategy can be harnessed to optimize the combined performance of human-machine teams by considering the distinct abilities of people and machines. The goal is to focus machine learning on problem instances that are difficult for humans, while recognizing instances that are difficult for the machine and seeking human input on them. We demonstrate in two real-world domains (scientific discovery and medical diagnosis) that human-machine teams built via these methods outperform the individual performance of machines and people. We then analyze conditions under which this complementarity is strongest, and which training methods amplify it. Taken together, our work provides the first systematic investigation of how machine learning systems can be trained to complement human reasoning.

Artificial intelligence promises to augment human capabilities and reshape companies, yet many organizations are finding that the results are falling far short of their expectations. This is frustrating but not surprising. Too often, companies try to implement AI without having a clear understanding of how the technology will interface with people.1 Over the past decade, we have done a number of studies to examine how companies use digital capabilities to become more competitive, including a recent study on human-machine collaboration in a cross-industry setting, where we sought to better understand the contexts in which organizations use particular digital systems.2 In this research, which included more than 20 case studies, we found that many organizations underestimated the value of teaming the predictive capabilities of algorithms with the expertise and intuitions of humans, especially in decision-framing.

The ability to create artificial intelligence (AI) capable of performing complex tasks is rapidly outpacing our ability to ensure the safe and assured operation of AI-enabled systems. Fortunately, a landscape of AI safety research is emerging in response to this asymmetry and yet there is a long way to go. In particular, recent simulation environments created to illustrate AI safety risks are relatively simple or narrowly-focused on a particular issue. Hence, we see a critical need for AI safety research environments that abstract essential aspects of complex real-world applications. In this work, we introduce the AI safety TanksWorld as an environment for AI safety research with three essential aspects: competing performance objectives, human-machine teaming, and multi-agent competition. The AI safety TanksWorld aims to accelerate the advancement of safe multi-agent decision-making algorithms by providing a software framework to support competitions with both system performance and safety objectives. As a work in progress, this paper introduces our research objectives and learning environment with reference code and baseline performance metrics to follow in a future work.

In the aftermath of the German U-boat campaign in the First World War, many in Europe and the United States argued that submarines were immoral and should be outlawed. The British Admiralty supported this view, and as Blair has described, even offered to abolish their submarine force if other nations followed suit. While British proposals to ban submarines in 1922 and 1930 were defeated, restrictions on their use where imposed that mandated that submarines could not attack a ship until such ships crews and passengers were placed in safety. This reaction to the development of a new means of war is illustrative of the type of ethical and legal challenges that must be addressed as military organizations adopt greater human-machine integration. This article is the final of three that examines the key aspects of human-machine teaming. In the first, I examined the rationale for human-machine teaming through seven'propositions'. The secondarticle examined three forms of human-machine teaming that military organizations might adopt in a closer integration of humans and machines.

This article discusses verification and validation (V&V) of autonomous systems, a concept that will prove to be difficult for systems that were designed to execute decision initiative. V&V of such systems should include evaluations of the trustworthiness of the system based on transparency inputs and scenario-based training. Transparency facets should be used to establish shared awareness and shared intent between the designer, tester, and user of the system. The transparency facets will allow the human to understand the goals, social intent, contextual awareness, task limitations, analytical underpinnings, and team-based orientation of the system in an attempt to verify its trustworthiness. Scenario-based training can then be used to validate that programming in a variety of situations that test the behavioral repertoire of the system. This novel method should be used to analyze behavioral adherence to a set of governing principles coded into the system.